Towards Sociable Virtual Humans: Multimodal Recognition of Human Input and Behavior

Towards Sociable Virtual Humans: Multimodal Recognition of Human Input and Behavior

—One of the biggest obstacles for constructing effective sociable virtual humans lies in the failure of machines to recognize the desires, feelings and intentions of the human user. Virtual humans lack the ability to fully understand and decode the communication signals human users emit when communicating with each other. This article describes our research in overcoming this problem by develop...

Behavior Generation for Interpersonal Coordination with Virtual Humans on Specifying, Scheduling and Realizing Multimodal Virtual Human Behavior Herwin

Behavior Generation for Interpersonal Coordination with Virtual Humans: on Specifying, Scheduling and Realizing Multimodal Virtual Human Behavior

Multimodal expression in virtual humans

This work proposes a real-time virtual human multimodal expression model. Five modalities explore the affordances of the body: deterministic, non-deterministic, gesticulation, facial and vocal expression. Deterministic expression is keyframe body animation. Non-deterministic expression is robotics-based procedural body animation. Vocal expression is voice synthesis, through Festival, and parame...

Virtual agent multimodal mimicry of humans

This work is about multimodal and expressive synthesis on virtual agents, based on the analysis of actions performed by human users. As input we consider the image sequence of the recorded human behavior. Computer vision and image processing techniques are incorporated in order to detect cues needed for expressivity features extraction. The multimodality of the approach lies in the fact that bo...